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LONG-TERM  GOALS 

A  broad  partnership  of  institutions  is  collaborating  in  developing  and  demonstrating  the 
performance  and  application  of  eddy-resolving,  real-time  global  and  basin-scale  ocean  prediction 
systems  using  the  HYbrid  Coordinate  Ocean  Model  (HYCOM).  These  systems  are  to  be 
transitioned  for  operational  use  by  the  U.S.  Navy  at  both  the  Naval  Oceanographic  Office 
(NAVOCEANO),  Stennis  Space  Center,  MS,  and  the  Fleet  Numerical  Meteorology  and 
Oceanography  Center  (FNMOC),  Monterey,  CA,  and  by  NOAA  at  the  National  Centers  for 
Environmental  Prediction  (NCEP),  Washington,  D.C.  The  systems  will  run  efficiently  on  a  variety 
of  massively  parallel  computers  and  will  include  sophisticated,  but  relatively  inexpensive,  data 
assimilation  techniques  for  assimilation  of  satellite  altimeter  sea  surface  height  (SSH)  and  sea 
surface  temperature  (SST)  as  well  as  in-situ  temperature,  salinity,  and  float  displacement. 

The  partnership  represents  a  truly  broad  spectrum  of  the  oceanographic  community,  bringing 
together  academia,  federal  agencies,  and  industry/commercial  entities,  spanning  modeling,  data 
assimilation,  data  management  and  serving,  observational  capabilities,  and  application  of  HYCOM 
prediction  system  outputs.  The  institutions  participating  in  this  Partnership  have  long  histories  of 
supporting  and  carrying  out  a  wide  range  of  oceanographic  and  ocean  prediction-related  research 
and  data  management.  All  institutions  are  committed  to  validating  an  operational  hybrid-coordinate 
ocean  model  that  combines  the  strengths  of  the  vertical  coordinates  used  in  the  present  generation 
of  ocean  models  by  placing  them  where  they  perform  best.  This  collaborative  partnership  provides 
an  opportunity  to  leverage  and  accelerate  the  efforts  of  existing  and  planned  projects,  in  order  to 
produce  a  higher  quality  product  that  will  collectively  better  serve  a  wider  range  of  users  than 
would  the  individual  projects.  In  addition  to  operational  eddy-resolving  global  and  basin-scale 
ocean  prediction  systems  for  the  U.S.  Navy  and  NOAA,  respectively,  this  project  offers  an 
outstanding  opportunity  for  NOAA-Navy  collaboration  and  cooperation  ranging  from  research  to 
the  operational  level. 

This  effort  is  part  of  a  5-year  (FY04-08)  multi-institutional  National  Ocean  Partnership  Program 
(NOPP)  project  which  includes  the  Florida  State  University  (E.  Chassignet),  U.  of  Miami  (G. 
Halliwell,  M.  Iskandarani,  T.  Chin,  A.  Mariano,  Z.  Garraffo,  A.  Srinivasan),  NRL/STENNIS  (H. 
Hurlburt,  A.  Wallcraft,  J.  Metzger,  B.  Kara,  J.  Cummings,  G.  Jacobs,  H.  Ngodock,  C.A.  Blain,  P. 
Hogan,  J.  Kindle),  NAVOCEANO  (F.  Bub),  FNMOC  (M.  Clancy),  NRL/MONTEREY  (R.  Hodur, 
J.  Pullen,  P.  May),  N O AA/N CEP/MM AB  (H.  Tolman,  C.  Lozano),  NOAA/NOS  (F.  Aikman), 
NOAA/AOML  (C.  Thacker),  NOAA/PMEL  (S.  Hankin),  Planning  System  Inc.  (O.M.  Smedstad), 
NASA-GISS  (R.  Bleck),  SHOM  (R.  Baraille),  LEGI  (P.  Brasseur),  OPeNDAP  (P.  Cornillon),  U.  of 
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OBJECTIVES 

The  partnership  is  addressing  the  Global  Ocean  Data  Assimilation  Experiment  (GODAE)  objectives 
of  three-dimensional  (3D)  depiction  of  the  ocean  state  at  fine  resolution  in  real-time  and  provision 
of  boundary  conditions  for  coastal  and  regional  models.  It  will  also  provide  the  ocean  component 
and  oceanic  boundary  conditions  for  a  global  coupled  ocean-atmosphere  prediction  model.  It  will 
make  these  results  available  to  the  GODAE  modeling  community  and  general  users  on  a  24/7 
operational  basis  via  a  comprehensive  data  management  strategy. 

APPROACH  AND  WORK  PLAN 

HYCOM  development  is  the  result  of  collaborative  efforts  among  the  Florida  State  University, 
University  of  Miami,  and  the  Naval  Research  Laboratory  (NRL)  as  part  of  the  multi-institutional 
HYCOM  Consortium  for  Data-Assimilative  Ocean  Modeling.  This  effort  was  funded  by  the 
National  Ocean  Partnership  Program  (NOPP)  to  develop  and  evaluate  a  data-assimilative  hybrid 
isopycnal-sigma-pressure  (generalized)  coordinate  ocean  model  (Bleck,  2002;  Chassignet  et  al., 
2003;  Halliwell,  2004).  More  details  can  be  found  at  http://www.hycom.org. 

While  HYCOM  is  a  sophisticated  model,  including  a  large  suite  of  physical  processes  and 
incorporating  numerical  techniques  that  are  optimal  for  dynamically  different  regions  of  the  ocean, 
data  assimilation  is  still  essential  for  ocean  prediction  a)  because  many  ocean  phenomena  are  due  to 
flow  instabilities  and  thus  are  not  a  deterministic  response  to  atmospheric  forcing,  b)  because  of 
errors  in  the  atmospheric  forcing,  and  c)  because  of  ocean  model  imperfections,  including 
limitations  in  resolution.  One  large  body  of  data  is  obtained  remotely  from  instruments  aboard 
satellites.  They  provide  substantial  infonnation  about  the  ocean’s  space-time  variability  at  the 
surface,  but  they  are  insufficient  by  themselves  for  specifying  the  subsurface  variability.  Another 
significant  body  of  data  is  in  the  form  of  vertical  profiles  from  XBTs,  CTDs,  and  profiling  floats 
( e.g .,  ARGO).  Even  together,  these  data  sets  are  insufficient  to  determine  the  state  of  the  ocean 
completely,  so  it  is  necessary  to  exploit  prior  knowledge  in  the  form  of  statistics  determined  from 
past  observations  as  well  as  our  understanding  of  ocean  dynamics.  We  combine  all  sources  of 
information  synergistically  to  produce  the  best  possible  depiction  of  the  evolving  ocean.  Several 
techniques  for  assimilating  data  into  HYCOM  are  either  in  place  or  under  development. 

RESULTS 

We  report  here  on  progress  made  in  FY09  (one-year  extension  of  the  original  award)  by  QinetiQ 
North  America  in  collaboration  with  FSU  on  the  HYCOM  1/12°  global  ocean  prediction  system  and 
by  FSU  on  the  management  of  ocean  model  outputs. 

a)  Global  l/12°ocean  prediction  system 

NCODA  Implementation: 

The  Navy  Coupled  Ocean  Data  Assimilation  (NCODA),  (Cummings,  2005),  is  the  assimilation 
component  in  HYCOM.  The  NCODA  system  is  a  fully  three-dimensional  multivariate  optimum 
interpolation  system.  The  three-dimensional  ocean  analysis  variables  include  temperature,  salinity, 
geopotential,  vector  velocity  components  and  in  support  of  HYCOM,  a  new  analysis  variable  that 
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corrects  the  model  layer  pressure  of  the  hybrid  vertical  coordinates.  An  analysis  of  sea  ice 
concentrations  is  also  a  part  of  the  NCODA  analysis.  The  NCODA  horizontal  correlations  are 
multivariate  in  geopotential  and  velocity,  thereby  permitting  adjustments  (increments)  to  the  mass 
fields  to  be  correlated  with  adjustments  to  the  flow  fields.  The  velocity  adjustments  are  in 
geostrophic  balance  with  the  geopotential  increments,  and  the  geopotential  increments  are  in 
hydrostatic  agreement  with  the  temperature  and  salinity  increments.  Either  the  Cooper  and  Haines 
(1996)  technique  or  synthetic  T  &  S  profiles  (Fox  et  ah,  2002)  can  be  used  for  downward  projection 
of  SSH  and  SST.  Experiments  using  the  two  methods  showed  that  the  synthetic  approach  gave  the 
best  results  when  the  model  was  compared  to  unassimilated  temperature  profiles.  Two  changes  to 
the  assimilation  methodology  were  implemented.  One  modification  is  to  increase  the  time  window 
of  the  in-situ  profile  observations  used  in  assimilation  from  ±12  hours  to  -12  days  through  ±12 
hours  of  the  analysis  time.  The  longer  time  window  allows  such  observations  to  have  a  greater 
influence  on  successive  ocean  analyses,  but  the  depth-varying  weight  given  to  each  profile 
diminishes  the  older  it  becomes.  Under  the  old  methodology,  a  profile  is  only  incorporated  into  the 
analysis  one  time.  If  on  the  next  day  an  altimeter  pass  indicated  a  significant  SSH  change,  a 
synthetic  MOD  AS  profile  would  be  generated  that  would  essentially  erase  the  previous  day’s 
observation.  An  observed  profile  should  more  accurately  reflect  the  subsurface  T  and  S  structure 
than  a  synthetic  profile,  even  if  it  is  a  few  days  old,  Thus  observed  profiles  are  now  favored  over 
synthetic  profiles.  A  second  modification  to  the  assimilation  methodology  is  the  use  of  the  ocean 
model’s  mixed  layer  depth  (MLD)  to  modify  and  more  accurately  represent  the  vertical  structure  of 
the  synthetic  MODAS  profiles,  which  tend  to  have  a  shallow  MLD  bias.  Below  the  forecast  MLD 
the  modified  synthetic  relaxes  back  to  the  original  T  and  S  profile  using  a  10  m  relaxation  scale. 
This  leads  to  a  sharper  transition  at  the  base  of  the  mixed  layer. 

The  NCODA  analysis  is  split  into  the  seven  sub  domains  shown  in  Figure  1 .  The  regions  covering 
the  Mercator  part  of  the  model  domain  follows  the  natural  land  boundaries  between  the  ocean 
basins.  Each  domain  is  run  on  a  fraction  of  the  total  number  of  available  processors  so  that  each 
domain  finishes  at  about  the  same  time. 


mean  sea  surface  height  2007-2008  (72.3) 


Figure  1.  The  figure  shows  the  SSH  field  from  the  1/12°  Global  HYCOM  domain  with  the  7 
regions  where  the  Mercator  part  of  the  domain  is  split  into  the  three  ocean  basins  using  the  natural 
land  boundaries. 
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1/12°  Global  HYCOM: 


The  1/12°  global  HYCOM  source  code  has  had  two  significant  modifications.  The  first  is  a  change 
in  the  vertical  remapping  scheme  (remapping  between  z-levels  and  hybrid  coordinates)  within  the 
hybrid  grid  generator  (hybgen).  Previously  the  piecewise  parabolic  method  (PPM)  was  used  but  this 
is  changed  to  a  weighted  essentially  non-oscillatory  (WENO)-like  PPM  scheme  for  increased 
accuracy.  The  second  change  is  in  hybgen  when  there  is  a  too-light  layer  on  top  of  a  too-dense 
layer,  i.e.  when  both  layers  would  like  to  gain  mass  at  the  expense  of  each  other.  Previously  hybgen 
chose  each  layer  half  of  the  time,  but  in  practice  the  thicker  of  the  two  layers  tended  to  gain  mass 
and  over  time,  the  thinner  layer  tended  to  become  very  thin  and  stay  that  way.  Now  the  thinner  of 
the  two  layers  always  gains  mass  from  the  thicker  layer,  greatly  reducing  this  tendency  for  layers  to 
collapse. 

RU27  (blue)  vs.  global  HYCOM  (red) 


Figure  2.  RU27  glider  path  (black  line)  and  150  m  averaged  current  vectors  (RU27  -  blue  arrows, 
HYCOM  -  red  arrows)  sampled  once  per  day  nearest  00Z  (top  panel).  One  hundred  fifty  meter 
depth  averaged  speed  (cm/s)  as  a  function  of  time  (bottom  panel)  using  the  same  color  scheme. 

The  validation  tasks  performed  this  year  include  a  temperature  vs.  depth  error  analysis  and  an 
acoustical  proxy  (mixed  layer  depth,  sonic  layer  depth,  below  layer  gradient  and  deep  sound 
channel)  error  analysis.  In  addition,  it  focused  on:  a)  the  two  systems  as  being  providers  of 
boundary  conditions  (BCs)  to  a  higher  resolution  nested  Relocatable  NCOM,  b)  an  examination  of 
the  14-day  forecast  skill  relative  to  climatology  and  persistence  of  the  temperature  (T)  versus  depth, 
c)  an  examination  of  the  14-day  forecast  skill  of  the  acoustical  proxy  variables,  d)  an  examination 
of  14-day  forecast  skill  of  sea  surface  height  (SSH)  and  sea  surface  temperature  (SST),  e)  a 
comparison  of  the  system's  ability  to  track  drifting  buoys  and  f)  the  ability  of  the  systems  to 
simulate  ocean  velocity  against  unassimilated  current  measurements.  These  comparisons  are 
performed  using  results  from  a  new  hindcast  with  the  above  mentioned  improvements  to  the  system 
as  well  as  results  from  the  real  time  system.  An  example  of  the  comparison  of  the  upper  ocean 
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velocity  field  is  shown  in  Figure  3  and  Table  1.  During  the  summer/fall  of  2008  and  2009,  the 
Coastal  Ocean  Observing  Lab  at  Rutgers  University  attempted  two  trans-Atlantic  flights  using 
Slocum  gliders.  These  began  off  the  U.S.  East  coast  and  traveled  eastward  using  the  Gulf  Stream  as 
a  tail  current  to  speed  their  progress  (http://rucool.marine.rutgers.edu/atlantic/index.html).  The 
glider  shown  here  is  RU27  which  began  on  27  April  2009.  The  glider  returns  average  currents  in 
the  top  150  m  of  the  ocean.  Because  of  the  shallow  dive  extent,  T  and  S  were  not  assimilated  into 
the  HYCOM  system.  The  glider  makes  several  dives  per  day  and  the  analysis  here  uses  glider 
observations  closest  to  00Z  which  are  compared  against  the  nearest  model  gridpoint  also  at  00Z  and 
averaged  over  comparable  depths.  Figure  2  depicts  the  glider  paths  and  current  vectors  sampled 
once  per  day  and  the  average  speed.  Vector  correlation  [u-v  /  sqrt((u-u)  *  (v-v))]  is  shown  in  Table 
1 


Vector 

correlation : 

RU27 

vs.  HYCOM 

May 

June 

July 

August 

September 

All 

.48 

.48 

.  61 

.19 

.56 

.47 

Table  1:  Vector  correlation  between  the  unassimilated  glider  and  HYCOM. 


sea  surf. 
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Figure  3.  The  SSH  field  from  the  real  time  1/12°  Global  HYCOM  domain  (a)  on  3  December  2009 
in  the  Kuroshio  region  and  (b)  on  4  December  2009  in  the  Gulf  Stream  region.  The  black  line 
represents  the  frontal  analysis  of  MCSST  observation  performed  at  the  Naval  Oceanographic 
Office.  A  black  line  represents  data  more  than  four  days  old. 

The  pre-operational  nowcast/forecast  system  using  the  1/12°  global  HYCOM  continues  to  run  in 
real-time.  It  started  running  in  near  real  time  on  22  December  2006  and  in  real-time  on  16  February 
2007.  The  real  time  system  was  moved  to  the  new  Cray  XT5  at  the  Naval  Oceanographic  Office 
and  is  now  running  on  619  processors  (24  nodes)  using  a  part  of  the  operational  allocation  on  the 
machine.  The  daily  run  consists  of  a  5  day  hindcast  and  a  5  day  forecast  and  takes  about  ~14  wall 
clock  hours.  The  NCODA  analysis  is  performed  at  18Z  and  the  model  is  incrementally  updated  over 
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the  next  6  hours  so  that  the  data  from  the  1 8Z  analysis  is  in  the  model  at  the  00Z  nowcast.  The  real¬ 
time  web  page  showing  the  results  from  the  system  can  be  found  on  the  HYCOM  consortium  web 
page,  http://www.hycom.org.  The  model  output  is  also  available  through  this  web  site.  The  model 
is  routinely  compared  to  both  independent  observations  and  observations  that  are  used  in  the 
assimilation.  An  example  of  an  independent  observation  comparison  can  be  seen  in  Figure  3a-b. 
These  figures  show  the  SSH  in  the  Kuroshio  and  Gulf  Stream  region,  respectively.  The  white/black 
line  represents  the  frontal  analysis  of  MCSST  observations  performed  at  the  Naval  Oceanographic 
Office.  A  black  line  represents  data  more  than  four  days  old.  The  model  is  able  to  accurately  depict 
the  frontal  position  in  these  regions,  showing  most  of  the  meanders  indicated  in  the  observations. 

Additional  information  about  the  validation  of  the  nowcast/forecast  system  can  be  found  in  Metzger 
et  al.  (2008)  and  Metzger  et  al.  (2009). 

CICE: 

The  coupling  between  HYCOM  and  the  Los  Alamos  ice  model  (CICE)  has  been  tested  in  an  Arctic 
Cap  version  of  HYCOM.  CICE  and  HYCOM  are  coupled  via  the  Earth  System  Modeling 
Framework  (ESMF).  The  NCODA  sea  ice  analysis  was  used  to  update  the  ice  concentration  in  the 
ice  model.  Experiments  showed  that  the  assimilation  of  the  analyzed  ice  concentration  had  a  very 
positive  impact  on  the  positioning  of  the  ice  edge.  Figure  4a-b  shows  an  example  from  a  hindcast 
experiment.  In  figure  2a  the  ice  thickness  for  20  September  2007  is  shown.  The  black  line  is  an 
independent  ice  edge  analysis  performed  at  the  National  Ice  Center  using  SSMI  observations.  The 
minimum  ice  concentration  observed  in  the  fall  of  2007  is  clearly  represented  in  the  model.  Figure 
4b  shows  the  ice  concentration  on  3  March  2008.  The  ice  has  grown  back  and  matches  the  observed 
ice  edge. 


ARCc0.08-02.7  Ice  Concentration:  20070920 


ARCc0.08-02.7  Ice  Concentration:  20080303 


(a)  (b) 

Figure  4.  Ice  concentration  from  CICE  is  shown.  The  black  line  represents  the  ice  edge  analysis 
performed  at  the  National  Ice  Center  using  SSMI  observations,  (a)  shows  the  ice  concentration  on 
20  September  2007  and  (b)  the  ice  concentration  on  3  March  2008. 
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b)  Data  management: 

The  main  goal  was  to  develop  and  implement  a  comprehensive  data  management  and  distribution 
strategy  that  allows  easy  and  efficient  access  to  HYCOM-based  ocean  prediction  system  outputs  to 
(a)  coastal  and  regional  modeling  sites,  (b)  to  the  wider  oceanographic  and  scientific  community 
including  climate  and  ecosystem  researchers,  and  (c)  the  general  public  especially  students  in 
middle  and  high  schools.  The  basic  idea  consists  of  the  setup  of  a  web  server  that  acts  as  a  gateway 
to  backend  data  management,  distribution  and  visualization  applications.  These  applications  enable 
end  users  to  obtain  a  broad  range  of  services  such  as  browsing  of  datasets,  gif  images,  NetCDF 
files,  FTP  request  of  data  etc.  The  HYCOM  Data  Sharing  System  is  built  upon  two  existing 
software  components:  the  Open  Project  for  a  Network  Data  Access  Protocol  (OPeNDAP)  and  the 
Live  Access  Server  (LAS).  These  tools  and  their  use  to  distribute  the  data  are  described  below.  In 
the  current  setup,  the  OPeNDAP  component  provides  the  middleware  necessary  to  access 
distributed  data,  while  the  LAS  functions  as  a  user  interface  and  a  product  server.  The  abstraction 
offered  by  the  OPeNDAP  server  also  makes  it  possible  to  define  a  virtual  data  set  that  LAS  will  act 
upon,  rather  than  physical  files.  An  OPeNDAP  “aggregation  server”  utilizes  this  approach  to 
append  model  time  steps  from  many  separate  files  into  virtual  datasets. 

The  HYCOM  Data  service  has  been  in  operation  for  the  last  five  years  and  has  seen  a  steady 
increase  in  the  user  base.  A  Storage  Area  Network  (SAN)  consisting  of  three  servers  consisting  of  4 
AMD  dual  core  Opteron  CPUs  attached  over  fibre  channel  to  130  TB  of  SATA  based  storage  has 
been  recently  purchased.  In  this  system,  the  servers  are  configured  in  a  high  availability  mode  and 
runs  a  Global  file  system  which  will  allow  concurrent  read/write  requests  from  all  the  attached  24 
CPU’s.  The  server  design  and  overall  operational  philosophy  includes  high  availability  and  high 
reliability  features  to  allow  for  uninterrupted  use. 

IMPACT/APPLICATIONS 

Three-dimensional  (3D)  depiction  of  the  ocean  state  at  fine  resolution  in  real-time  and  provision  of 
boundary  conditions  for  coastal  and  regional  models  in  the  context  of  the  Global  Ocean  Data 
Assimilation  Experiment  (GODAE). 

TRANSITIONS 

The  1/12°  global  HYCOM  system  is  running  in  real  time,  and  was  transitioned  to  NAVOCEANO  at 
the  end  of  FY08  with  operational  testing  is  planned  in  FY10. 

RELATED  PROJECTS 

This  is  a  highly  collaborative  NOPP  project.  Additionally,  the  project  is  receiving  grants  of  super 
computer  time  from  the  DoD  High  Performance  Computing  Modernization  Office. 
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